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Hybrid Reality in the Internet of Things 
as an Environment for Transferring Knowledge 


Abstract 


The paper presents features and applications of hybrid reality technology and the 
Internet of Things. The analysed thesis states that such a solution is a favourable 
environment for the dissemination of knowledge through the concept of interaction 
of objects of postulated environment with the theoretical model of knowledge 
objects. The article discusses an initial version of the potential information 
system method. The method, in the form of a mathematical formula, makes it 
possible to evaluate objects in the postulated space. The article presents the case 
study implementing a similar environment in the service of advanced industrial 
equipment. The results of the analysis indicate a high potential of the postulated 
solution, which requires further refinement and research. At the same time, the 
practice of the implementation case points to technological and organisational 
threats which should be neutralised to ensure a high probability of the project’s 
success. 
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Due to the development of new interaction methods with devices, it seems 
important to ask questions about the possibilities of using technology in broadly 
understood education. The aim of this article is to present a new measure of the 
objects ability to accumulate knowledge in the light of the conditions that new 
technologies (hybrid reality and Internet of Things) bring, and also to present a 
case study based on the technologies mentioned in the article. The article discusses 
the features of augmented reality, presenting the application of an additional visual 
layer to the observed reality together with the concept of the Internet of Things as 
a network of people, processes, data, and objects. The concept of using artificial 
intelligence supporting the transfer of knowledge as well as the classification and 
valuation of objects in such an environment that combines the above-mentioned 
technologies will also be presented. 


Classification of Artificial Reality 


When we analyse the history of the development of user interfaces for informa- 
tion systems, we notice a constant trend to approach the methods of communi- 
cation and interaction between man and machine, reflecting the natural capabilities 
of the human senses and the way in which the human functions in the real world. 
The objects in the paper are understood as physical objects (devices, people, 
creatures, places, symbols) and digital objects (entities, properties, activities and 
processes). 


Augmented Reality 

Augmented Reality (AR) is based on the integration of digital information 
into the user's real-time environment. Augmented Reality does not create 
a new, full, virtual 3D world (for it deals with virtual reality), but it extends 
and complements the one we know. In other words, it converges and connects 
digital and physical worlds into a uniform visual impression (Zur, 2013). It is 
important to preserve the physics of objects and proper orientation in relation to the 
reference system as well as other spatial relations. AR applications are written in 
3D programs that allow programmers to associate animation or contextual digital 
information in a computer program with an AR tag in the real world. When an 
application or browser plug-in of a computing device receives digital information 
from a known tag, it begins to execute the tag code and places the correct image 
or images. 
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Virtual Reality 

Virtual Reality (VR) is a fully computer-generated simulation of a 3D image 
or environment with which it is possible to interact in a seemingly real or physical 
way. It is also the use of information technology to create the effect of an interactive 
three-dimensional world in which each object has the sense (property) of being 
present in this space (Bryson, 1999). The user is equipped with special electronic 
equipment, such as a helmet with a built-in screen or/and gloves with sensors. 
It is a completely artificially created space. The interaction of the observer and 
objects that most often reflect the physics of such interactions in the real space are 
important. Direct references to real space usually do not occur at all. 


Mixed Reality 

The hybrid reality (Mixed Reality) is a combination of the real world and the 
virtual one, in order to create a new environment where physical and digital objects 
can interact. It, in fact, a combination of augmented and virtual reality (de Souza 
e Silva & Sutko, 2009). Objects superimposed on a real image may have more 
opportunities to enter into a variety of relationships with each other and with an 
observer. In a perfect world, devices used to apply artificial reality to a perceived 
image should be transparent. The Google Glass market experiment was the first 
step towards the dissemination of this mode of operation. On the glasses, there 
was a picture that provided contextual information needed by the user. There was 
no current specific spatial relations and extensive interactions in this solution, 
but the platform constituting the foundations of such a solution in the future was 
commercialised and made available to a wide range of users. The product has 
not gained much consumer recognition (Reynolds, 2015). Its implementation for 
technical reasons differed somewhat from the assumption of transparency. Glasses 
definitely differed from other frames. In addition, they caused asymmetries in 
the face image, which in the medical sciences is often described as a symptom of 
dysmorphia indicating various diseases. In social communication, the image of the 
face and the ability to read information from it is extremely important. Devices 
that allow the use of AR should not build a barrier in this regard. 

Assuming that the physical layer should be transparent, an interesting solution 
seems to be the concept of contact lenses with a built-in liquid crystal display. In 
daylight, the backlight of such a matrix could be passive. It is a device that uses 
a small amount of energy so it could be powered through a communication channel, 
for example, RFID 5.8 GHz. It would not only provide data exchange, but also 
provide electricity by induction (Adhikari, 2016). As for the interaction of the 
observer with the objects superimposed in space, the sensor could be supplemented 
by sensors glued to nail plates. Such a solution would ensure precise interpretation 
of various gestures. The operating procedure in hybrid reality mode is as follows: 
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- 


. Identification of the object through а marker. 

2. The graphic representation in the added reality layer is displayed by means of 
appropriate devices imposing an artificial image on the real one. 

3. The object maintains the appropriate spatial relations. 

4. The object presents various possibilities of interaction as part of the internet 
formula of the universe. 

5. The object exhibits spatial relations with the observer through various sensors 

that map the observer’s movements. 


Internet of Things and Connection with Artificial Reality 

The Internet of Things is comprised of three kinds of connections: machine- 
machine, human-machine and human-human. For the Internet of Things to become 
a reality, it is necessary to develop and implement technologies that, on the one 
hand, will allow unconnected elements of the physical world to be connected 
together, and on the other, they will enable the use of a huge amount of data 
generated by them. Nowadays, the most common method of connecting physical 
objects with internet is QR Code (Koreňová & Hvorecký, 2018). Meanwhile, it is 
estimated today that 12.5 billion devices are connected to the Internet, which 
is only 1 percent of objects that could potentially be plugged into the network. It is 
estimated that in 2020, between 25 and 50 billion devices will be connected to 
the network (Jaslan, 2015). The integration of data, processes, people and devices 
(Figure 1) along with artificial intelligence opens up many new opportunities in 
the area of acquiring knowledge and using it (Ravi & Shalinie, 2020). Processes 
that combine existing data with objects and individuals ultimately form the so- 
called intelligent spaces (Brachman, 2017), collections of beings of various types, 
which interacting with each other generate information and knowledge resources. 


и N 


Processes 


Internet of 
All Things 


Things 


NS d 
Figure 1. The Internet of Things 


Source: the authors’ own study based on Cisco materials. 
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The subject of this study is to present the possibilities of combining hybrid 
reality and the Internet of Everything in the context of knowledge management. 
Merill, Li and Jones identify four types of knowledge objects: entities, properties, 
activities and processes: “Beings are represented by physical objects such as 
devices, people, creatures, places, symbols, etc. Properties mean measurable and 
irrational attributes of beings” (Merill, Li, & Jones, 1991). Activities are actions 
that the learner can undertake towards beings. Processes, on the other hand, 
correspond to events that change the values and properties of beings and are 
triggered by activities or other processes. 

The above classification is very similar to the list of components of the Internet 
of Things. It seems natural, therefore, to extrapolate this classification in terms of 
technology in which knowledge facilities can be given new properties. To automate 
the process of acquiring knowledge in objects, one can refer to the notion of 
inheritance known from computer science. The classes of objects are indispensable 
for this. A given physical object, for example, a refrigerator, would belong to 
the classes of knowledge: refrigeration, mechatronics, etc., then sub-classes: 
refrigerators, a specific line of refrigerators and a specific type and specimen. 
After identifying such an object in a hybrid reality by the use of the Internet of 
Things, that is, through the process of combining data and objects, the knowledge 
would be inherited from the classes to which it belongs. A physical object that does 
not have to be a device at all (see, for example, the field of wheat and the definition 
of being within the framework of knowledge objects in the further part of this 
article) may not have the ability to accumulate knowledge or even record data in 
itself. In order for the inheritance and “learning” of the object to take place, it is 
necessary to have a virtual representation of the object on the Internet of Things 
in the form of a specific avatar or simulator. 

In the proposed concept, explained above, it seems necessary to introduce an 
application layer in which an avatar of an individual object is created ad hoc and 
based on inheritance from its classes and then through interactions with other 
objects, such as technical documentation, instructions or courses. If we allow the 
possibility of incorporating advanced artificial intelligence into this layer, the 
appropriate connections could form spontaneously. It is necessary in this case 
to have a fairly complete semantic network, which identifies through metadata 
within the ontology: first, belonging to classes, second, making the contents of 
the knowledge object or its avatars readable for devices, which allows verification 
of the usefulness of establishing interactions. 

The authors of this study suggest the introduction of a new value: the measure 
of the accumulation of object knowledge. The indicator (measure) is most often 
understood as a number expressing the level of a given phenomenon. The most 
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important feature of the indicator is the comparability of its value, allowing to 
determine the position of a given object compared to other objects (Rogala & 
Rycharski, 2006). In this case, it will be the term saturation of an object with 
knowledge in the context of others as a function of interaction with other objects 
and inheritance in the classroom. It is, therefore, a function of several features, also 
called diagnostic variables. There is a multiplicity of attributes so we can define the 
meter as synthetic or aggregate. The characteristics that influence the index can 
be included in the set of stimulants, that is, variables whose higher value indicates 
a higher level of the phenomenon (object) and thus works in a way that stimulates 
development (Kompa, 2009). 

The measure of the accumulation of knowledge by the object is postulated to 
refer to the number of interactions with other objects, their quality and weight, plus 
the value of the measure of accumulation of knowledge resulting from inheritance 
in classes. 


Am = 2i=1 94% + DA,, = 27-,9;а + р 


m 


The A,, measure is the value corresponding to the accumulation of knowledge of 
the object m. 

q, — quality of i-th interaction expressed in time, assessed оп a scale or based 

on the analysis of measurable results, 

а; — a measure of the accumulation of knowledge of the i-th object with which 

the interaction took place, 

D — measure of the accumulation of knowledge resulting from inheritance in 

classes. 

The measure of the accumulation of object's knowledge is nothing but a 
measurable property of being (compare with the classification of knowledge 
objects), and interaction is an activity leading to the process of transforming the 
properties of an object (being). It seems possible to develop a comprehensive 
standard, method and system using these technologies, which would become 
common on a similar scale as Google search, but limited to the aspect of acquiring 
and accumulating the knowledge of all entities. The possibility of registering this 
type of transformations and building the weight of objects in the context of their 
importance as objects of knowledge would give rise to new possibilities. A network 
that fosters the development of knowledge can arise spontaneously. In the context 
of the presented method, it is postulated to consider two levels of interaction, the 
first one — associated with the area of skills, and the second one — with broadly 
understood knowledge: 


IJREL.2020.6.1.09 р. 5/13 


Hybrid Reality in the Internet of Things аз an Environment for Transferring Knowledge 


The first level: 

— objects are presented in relation to the context of their use; 

— the object itself offers information and knowledge about itself in accordance 
with the context; 

— the object provides knowledge and information in the area of skills depending 
on the role of the being with which it interacts, for example, the fridge helps the 
user or the serviceman. 

The second level: 

— the object itself presents the areas of knowledge related to it based on the 
analysis of previously registered user's relations with other objects, their 
regularity of duration and detail. 

— in this way, the object can indicate areas of knowledge necessary to achieve a 
full understanding of both the object itself and the accompanying phenomena, 
for example, by offering courses, postgraduate studies or majors (this type of 
link would be possible within the inheritance of knowledge areas in classes). 

It is important to emphasise that each time it is not a physical object that leads 

a “conversation,” only the previously mentioned object avatar in the application 

layer of the postulated knowledge transfer environment, containing artificial 

intelligence that associates behavioural patterns and context and initiates 
interaction based on the evaluation of existing relationships. Figure 2 illustrates 
an example of human interaction with an object that is part of nature, namely, of 

a wheat field: 


Witaj, 


Zauważyłam, że szukałeś ostatnio 
na czym polega różnica pomiędzy 
wyglądem pszenicy a żyta... 

Więc, jestem pszenicg. Mito mi. 
Jestesmy z żytem jakby siostrami... 
Ale nie bliźniaczkami © Jeśli chcesz 


zobaczyć żyto, to jest niedaleko... 
Wyznaczyć Ci trasę? 


Figure 2. Interaction of a natural object with a human being 


Source: the authors’ own study (translation: “I have noticed that you have been searching recently 
for the difference between wheat and rye. I am wheat. It is nice to meet you. We are almost like sisters 
with rye. But not twins. If you wanted to see rye, it is not far from here... Would you like me to give 
you directions?”) 
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In the above example the user has a specific dialogue with an object that is 
identified in a hybrid reality through the techniques of the Internet of Things, 
connected with the avatar of its type of existence, and having a certain classification 
and inheriting knowledge interacting with other beings (rye) that match the profile 
of previous interactions human. The activity initiated and “proposed” by the object 
is aimed at starting a process that will lead to a change in the properties of being 
(human), that is, the state of knowledge in the area. 


Case Study 

Figure 3 presents an analysis of the case of the implementation of the system 
combining the Internet of Universes with Hybrid Reality, made by the Polish 
company PSC Transition Technologies for Fiat Chrysler Automotive. The aim 
of the project was to use the hybrid reality and the Internet of Things to support 
knowledge of the robotic production lines. The implementation was started with 
support for maintenance and maintenance through the loT solution combined 
with the use of Mixed Reality for employees of this department. The project 
was implemented with the use of a tablet and glasses for Augmented Reality as 
a user interface, which significantly improves the performance of daily tasks by 
employees. In everyday operations, the objects that are avatars of the essential 
components of machines used for car production are identified before employees” 
eyes. Facilities have multiple connections with sources of knowledge. Glasses 
can display the process of device replacement taking place in three dimensions, 
preserving spatial relations with reality. It is a simulation of three-dimensional 
objects in motion. The process is initiated by the action of interacting the serviced 
device with other related knowledge objects, such as instructions or technical 
specifications, presented contextually. 


The project, after extending it to all elements of robotics on new lines, reduces 
the operational costs of the service by automating and unifying service tasks and 
optimising the structure of the internal maintenance service as well as internal 
communication. As a consequence, it shortens the duration of service procedures 
by increasing their effectiveness. The completely unexpected but desired result 
of the project was the development of new purchasing procedures, so that the 
necessary elements needed to attach new machines and robots to the system 
were available along with the installation of production machines on the hall 
(e.g., relevant 3D CAD drawings of machines). On the basis of reports and post- 
implementation data, made available by PSC Transition Technologies and the 
interview with implementation participants, the authors of the study carried out a 
SWOT analysis of the applied solution. 
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Table 1 


SWOT analysis of the implementation case in FCA 


Strenghts 


$1 – employees receive one environment 
to obtain the necessary information and 
knowledge. 


S2 — the environment is very ergonomic 
and intuitive because access is carried 
out through activities simulating reality. 


S3 — all information and knowledge are 
given in context, which eliminates time- 
consuming searches. 


Opportunities 


01 – opportunity to better profit by 
reduction of service operational costs 
through automation and unification of 
service tasks. 


02 – opportunity for be flexible in labour 
market by shortening the time of adapting 
new employees to applicable procedures. 


ОЗ — opportunity for better cooperation 
with machinery producers by 
development of new purchase 
procedures, so that the necessary 
elements needed to connect new 
machines and robots to the system were 
available together with the installation of 
the machine on the hall (e.g., appropriate 
drawings of 3D CAD machines). 


Source: the authors” own study 


Weaknesses 


W1 — knowledge base available within 
one environment. 


W2 – lack of reference to employees’ 
habits regarding traditional sources of 
information and knowledge acquisition, 
including informal ones. 


W3 — dependence on technologies and 
specific devices that require expensive 
service and maintenance. 


Threats 


Т1 - a threat to production continuity in 
the event of technical failures. 


T2 — over-routine procedures that do not 
take informal knowledge, which is not 
collected in any way. 


T3 — non-adjustment of accessory 
manufacturers due to the lack of 
knowledge objects associated with the 
product, which increases the cost of 
adjustment. 


The components from the SWOT table were subjected by the authors to the 
analysis of their own impact using a three-point scale from 0 to 2 (no impact, weak, 
strong). Conclusions regarding the impact force were taken during the discussion 
with the authors of the post-implementation reports and on the basis of the analysis 


of their conclusions and the data cited. 
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Table 2 
Point analysis of the SWOT ties strength 
01 02 ОЗ Т1 Т2 ТЗ 

$1 2 2 0 1 2 
$2 
$3 2 2 0 0 1 
W1 0 0 0 0 1 1 
W2 1 1 0 0 2 1 
W3 0 0 1 2 0 1 


Source: the authors’ own study 


Point analysis of relationships from individual quarters of the Table 2: 
S/O = 11, S/T = 6, W/O = 3, W/T = 8 


The highest score of the first quadrant suggests that thanks to the new 
technology used, the company is dominated by strengths, and in its environment 
strongly associated with them — it is a strategy of strong expansion and development 
using both factors. At the same time, the high point score of the fourth quarter 
shows a certain paradox and commands exceptional attention while minimising 
the strength of links between weaknesses and external threats. 


Summary 


The publication presents a combination of hybrid reality technologies and the 
Internet of Things. The thesis attempts to establish that such a combination is 
a favourable environment for spreading knowledge. In the course of the study, 
it was possible to combine the concepts of the interaction of objects of the 
postulated knowledge transfer environment with the measure of the accumulation 
of knowledge objects. The initial version of the method is presented in the form 
of a mathematical formula that allows valuing objects in such space. The case of 
implementation of a similar environment in the service of advanced industrial 
equipment has been presented and analysed. In conclusion, one should note the 
high potential of the postulated solution, which requires further refinement and 
research. At the same time, the practice of the implementation case points to 
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technological and organisational threats which should be neutralised to ensure 
a high probability of the project's success. 
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Rzeczywistość hybrydowa w Internecie Wszechrzeczy 
jako środowisko do transferu wiedzy 


Streszczenie 


W publikacji przedstawiono cechy i zastosowania technologii rzeczywistości hybrydowej oraz 
Internetu Wszechrzeczy. Przeanalizowano tezę, że takie zestawienie stanowi korzystne środowisko 
do rozprzestrzeniania wiedzy poprzez koncepcję interakcji obiektów postulowanego środowiska 
z teoretycznym modelem obiektów wiedzy. Przedstawiono wstępną wersję metody potencjalnego 
systemu informatycznego. Metoda ta, w postaci formuły matematycznej umożliwia wartościowa- 
nie obiektów w postulowanej przestrzeni. Przedstawiono przypadek wdrożenia zbliżonego środo- 
wiska w praktyce serwisowej zaawansowanych urządzeń przemysłowych. Wyniki analiz wskazują 
na wysoki potencjał postulowanego rozwiązania, które wymaga dalszego uściślania i badań. Jed- 
nocześnie praktyka przypadku wdrożenia wskazuje na zagrożenia technologiczne i organizacyjne, 
które należy neutralizować, aby zapewnić wysokie prawdopodobieństwo sukcesu przedsięwzięcia. 


Słowa kluczowe: Rzeczywistość wirtualna, rzeczywistość rozszerzona, zarządzanie wie- 
dzą, akumulacja wiedzy 
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Гибридная реальность в интернете вещей 
как среда для передачи знаний 


Аннотация 


В статье представлены особенности и применения технологий гибридной реальности 
и Интернета вещей. В анализируемом тезисе утверждается, что такое решение является 
благоприятной средой для распространения знаний через концепцию взаимодействия 
объектов постулируемой среды с теоретической моделью объектов знаний. Представлена 
первоначальная версия метода потенциальной информационной системы. Этот метод в фор- 
ме математической формулы позволяет оценивать объекты в постулируемом пространстве. 
Представлено тематическое исследование внедрения аналогичной среды в обслуживании 
современного промышленного оборудования. Результаты анализа указывают на высокий 
потенциал постулируемого решения, что требует дальнейшей доработки и исследования. 
В то же время, практика реализации проекта указывает на технологические и организаци- 
онные угрозы, которые следует нейтрализовать, чтобы обеспечить высокую вероятность 
успеха проекта. 


Ключевые слова: Виртуальная реальность, дополненная реальность, управление 
знаниями, накопление знаний 
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Realidad híbrida en Internet de todas las cosas 
Como un ambiente para transferir conocimiento 


Resumen 


El documento presenta características y aplicaciones de la tecnología de realidad híbrida 
e Internet de todas las cosas. La tesis analizada establece que dicha solución es un entorno favorable 
para la difusión del conocimiento a través del concepto de interacción de los objetos del entorno 
postulado con el modelo teórico de los objetos de conocimiento. Se presenta una versión inicial 
del método potencial del sistema de información. Este método, en forma de fórmula matemática, 
permite evaluar objetos en el espacio postulado. Se presenta el estudio de caso que implementa un 
entorno similar al servicio de equipos industriales avanzados. Los resultados del análisis indican 
un alto potencial de la solución postulada, que requiere un mayor refinamiento e investigación. 
Al mismo tiempo, la práctica del caso de implementación apunta a amenazas tecnológicas y or- 
ganizativas que deben neutralizarse para garantizar una alta probabilidad de éxito del proyecto. 


Palabras clave: Realidad virtual, realidad aumentada, gestión del conocimiento, acumulación 
de conocimiento 
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